Skip to content

chore: centralize URL resolution, subdirectory routing, and deployment fixes (PR #243 fixes)#244

Closed
sheepdestroyer wants to merge 20 commits into
masterfrom
chore/learn-deployment-guidelines
Closed

chore: centralize URL resolution, subdirectory routing, and deployment fixes (PR #243 fixes)#244
sheepdestroyer wants to merge 20 commits into
masterfrom
chore/learn-deployment-guidelines

Conversation

@sheepdestroyer

@sheepdestroyer sheepdestroyer commented Jul 8, 2026

Copy link
Copy Markdown
Owner

This PR resolves review comments from PR #243, introduces centralized URL/domain resolution, and refactors base URL calculations.

Key Changes

  1. Centralized Base URL Resolution & Deduplication:
    • Extracted all URL parsing, validation, and domain-checking logic from get_dashboard into a small, testable utility function resolve_external_urls(request: Request) -> tuple[str, str, str] in router/main.py.
    • Used PUBLIC_BASE_URL (passed directly to the llm-triage-router container) as the primary source of truth, fallbacking to BASEURL/BASE_URL/request.base_url securely to prevent configuration drift between shell scripts and Python.
  2. Subdirectory Routing & Dynamic Resolution:
    • Router dashboard URLs dynamically map /llm-routing/litellm, /llm-routing/langfuse, and /llm-routing/llama/ subdirectories depending on request headers/domain.
    • Fixed LiteLLM SERVER_ROOT_PATH and added PROXY_BASE_URL behind HAProxy.
  3. Configurable Domain:
    • Extracted the production domain configuration to ROUTING_DOMAIN env variable in router/main.py (default: vendeuvre.lan).
  4. LiteLLM Health Probes:
    • Configured livenessProbe and readinessProbe to use direct in-pod endpoints (/ping and /health/readiness) rather than the reverse proxy paths.
    • Increased readiness probe delay to 45 seconds to prevent false startup failures.
  5. Sanitized Placeholders:
    • Reverted all hardcoded path (/mnt/DATA/boy/...) and user ID (1002) bindings back to generic WORKDIR_PLACEHOLDER, HOME_PLACEHOLDER, and RUN_USER_PLACEHOLDER templates, ensuring portability across multiple environments and restoring proper validation.

Summary by Sourcery

Centralize public/base URL handling across the router, deployment scripts, and pod configuration to support subdirectory routing and production domains while restoring portability placeholders and updating routing docs.

Bug Fixes:

  • Fix dashboard external links so they correctly resolve Langfuse, LiteLLM, and Llama.cpp URLs when running behind a reverse proxy subdirectory or custom domain, with strict domain validation to avoid incorrect host matching.
  • Correct LiteLLM health checks to avoid relying on reverse proxy paths and reduce false startup failures via increased readiness delay.

Enhancements:

  • Introduce a shared URL resolution helper for dashboard external links that respects PUBLIC_BASE_URL, request host, and a configurable ROUTING_DOMAIN with safe fallbacks for local development.
  • Derive PUBLIC_BASE_URL and LOCAL_BASE_URL in the startup script, inject them into pod.yaml (including PROXY_BASE_URL), and surface them in deployment logs for clearer entrypoints.
  • Re-enable the local-qwen-3.6 model in the LiteLLM configuration pointing to a llama.cpp instance on port 8081 and wire it back into model routing groups.
  • Update agent and README documentation to use generic paths, relative links, and a renamed NotebookLM knowledge base, and add a production deployment checklist.

Build:

  • Extend pod.yaml templating to use WORKDIR/HOME/RUN_USER placeholders instead of hardcoded user paths and to validate and fill new PROXY_BASE_URL and PUBLIC_BASE_URL placeholders.

Deployment:

  • Configure LiteLLM’s SERVER_ROOT_PATH and PROXY_BASE_URL env vars in the pod, adjust health probes to hit in-pod endpoints with a longer readiness delay, and document the production HAProxy-based deployment flow.

Documentation:

  • Generalize hardcoded absolute paths in README and internal agent docs to portable placeholders and add operational guidance for deployment, GitHub usage, and sudo password handling.

Summary by CodeRabbit

  • New Features

    • Added support for configurable public and local URLs in deployment and dashboard links.
    • Expanded the available free model roster and re-enabled a local Qwen model option.
  • Bug Fixes

    • Replaced hardcoded local paths and hostnames with placeholders and dynamic URL resolution for more reliable deployments.
    • Improved dashboard links so they point to the correct environment instead of localhost.
    • Adjusted readiness timing for a smoother startup experience.
  • Documentation

    • Updated setup and deployment guidance with clearer, more portable path references and a production checklist.

sheepdestroyer and others added 19 commits July 8, 2026 21:04
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…al_host/port resolution, fix agy startup SSH commands
…add fallbacks, avoid duplicate variables)
…PLACEHOLDER, HOME_PLACEHOLDER, and RUN_USER_PLACEHOLDER
@sourcery-ai

sourcery-ai Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Centralizes public/base URL resolution for the router and dashboards, introduces environment-driven domain/subdirectory routing, fixes deployment and health probe configuration, and restores portable placeholders in infra templates.

Sequence diagram for centralized dashboard URL resolution

sequenceDiagram
    actor User
    participant RouterApp as Router_app
    participant resolveExternal as resolve_external_urls

    User->>RouterApp: GET /dashboard
    RouterApp->>resolveExternal: resolve_external_urls(request)
    resolveExternal-->>RouterApp: langfuse_url, litellm_url, llama_url
    RouterApp-->>User: HTML dashboard (links use langfuse_url, litellm_url, llama_url)
Loading

File-Level Changes

Change Details Files
Centralized external dashboard URL resolution and domain-validated subdirectory routing for Langfuse, LiteLLM, and Llama.cpp.
  • Added resolve_external_urls(request) helper that derives scheme/host/netloc from PUBLIC_BASE_URL/BASEURL/BASE_URL or request.base_url with strict regex validation and ROUTING_DOMAIN checks.
  • Updated /dashboard handler to accept Request and inject resolved langfuse_url, litellm_url, and llama_url into the dashboard HTML links.
  • Implemented production subdirectory mapping (/llm-routing/langfuse, /llm-routing/litellm/ui, /llm-routing/llama/) with local-development fallbacks to localhost ports.
router/main.py
Aligned shell and pod configuration around PUBLIC_BASE_URL/LOCAL_BASE_URL and proxy paths, and wired these into pod templating.
  • Derived PUBLIC_BASE_URL and LOCAL_BASE_URL early in start-stack.sh with sensible defaults, normalization, and export to environment.
  • Extended pod.yaml rendering to require PROXY_BASE_URL_PLACEHOLDER and PUBLIC_BASE_URL_PLACEHOLDER and to compute PROXY_BASE_URL from PUBLIC_BASE_URL.
  • Updated deployment success messages to show public and local entry endpoints, dashboard URL, and LiteLLM Admin UI based on PUBLIC_BASE_URL instead of hardcoded localhost paths.
start-stack.sh
pod.yaml
Restored generic path/user placeholders and ensured pod templating validates them instead of hardcoded host-specific paths.
  • Replaced hardcoded absolute paths and UID-dependent run-user paths in pod.yaml with WORKDIR_PLACEHOLDER, HOME_PLACEHOLDER, and RUN_USER_PLACEHOLDER for volumes and mountPaths.
  • Changed render_pod_yaml in start-stack.sh to expect placeholder tokens rather than concrete paths and to substitute WORKDIR, HOME, and uid-derived run-user paths accordingly.
  • Updated README and agy_proxy comments to describe paths generically (e.g., /path/to/LLM-Routing, ~/.local/bin/agy) instead of user-specific ones.
start-stack.sh
pod.yaml
README.md
router/agy_proxy.py
Hardened LiteLLM deployment configuration and re-enabled a local Qwen model through llama.cpp.
  • Set SERVER_ROOT_PATH and PROXY_BASE_URL env vars for LiteLLM in pod.yaml to support routing under /llm-routing/litellm behind HAProxy.
  • Adjusted LiteLLM readinessProbe initialDelaySeconds from 10 to 45 to avoid premature failures while the service initializes.
  • Re-enabled local-qwen-3.6 in litellm/config.yaml with a new llama.cpp backend on port 8081 and updated dependent routing groups to include this model.
pod.yaml
litellm/config.yaml
Updated operational documentation and production deployment steps, including a checklist for the 'boy' host user and GitHub/agy operational notes.
  • Renamed the NotebookLM knowledge base references in .agents/AGENTS.md and added a detailed production deployment checklist for the boy user, including HAProxy, agy daemon, and verification commands.
  • Documented operational notes about systemd limitations, sudo password handling, fast vs full rebuild semantics, and GitHub CLI authentication.
  • Clarified README references for models.ini and systemd ProtectHome semantics and fixed memory_mcp.py link to be repo-relative.
.agents/AGENTS.md
README.md

Possibly linked issues

  • #LHF-03: This PR implements env-based, centralized URL resolution and updates hardcoded dashboard links, directly addressing LHF-03’s endpoint issue.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR introduces configurable public base URL support (ROUTING_DOMAIN/PUBLIC_BASE_URL) across start-stack.sh, pod.yaml, and router/main.py's dashboard, replaces hardcoded developer host paths with deployment placeholders, updates litellm model/fallback configuration, refreshes the free-models roster, and updates deployment/README documentation.

Changes

Public base URL and deployment placeholder changes

Layer / File(s) Summary
ROUTING_DOMAIN/PUBLIC_BASE_URL computation and rendering
start-stack.sh
Computes ROUTING_DOMAIN, PUBLIC_BASE_URL, LOCAL_BASE_URL, PROXY_BASE_URL; exports for template rendering, extends placeholder whitelist, replaces hardcoded path substitutions, and updates success messages.
pod.yaml placeholder substitution and env vars
pod.yaml
Adds SERVER_ROOT_PATH/PROXY_BASE_URL and PUBLIC_BASE_URL/ROUTING_DOMAIN env vars, replaces hardcoded /home/gpav and /run/user/1000 paths with placeholders, increases readiness probe delay.
Dynamic external URL resolution in dashboard
router/main.py
Adds LANGFUSE_HOST constant and resolve_external_urls() helper; updates /dashboard endpoint and Langfuse client init to use resolved URLs instead of hardcoded localhost links.
litellm model/fallback config for local-qwen-3.6
litellm/config.yaml
Re-enables local-qwen-3.6 in fallback chains for all core tiers and adds an active model_list entry with updated api_base/key/timeout.
Free models roster update
router/free_models_roster.json
Adds two models, removes one, and updates roster metadata.
Documentation and path-generalization updates
.agents/AGENTS.md, README.md, router/agy_proxy.py
Adds Production Deployment Checklist/Fresh Deploy Steps/Notes sections and generalizes absolute host paths to placeholders/relative references.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Dashboard as "router/main.py (/dashboard)"
  participant resolve_external_urls
  participant Env as "PUBLIC_BASE_URL/ROUTING_DOMAIN"

  Client->>Dashboard: GET /dashboard
  Dashboard->>resolve_external_urls: request
  resolve_external_urls->>Env: read PUBLIC_BASE_URL/ROUTING_DOMAIN
  alt public base URL configured and valid
    resolve_external_urls-->>Dashboard: langfuse_url, litellm_url, llama_url (public)
  else fallback to local
    resolve_external_urls-->>Dashboard: langfuse_url, litellm_url, llama_url (local)
  end
  Dashboard-->>Client: rendered dashboard with resolved links
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly reflects the main changes: URL resolution, subdirectory routing, and deployment-related fixes.
Docstring Coverage ✅ Passed Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/learn-deployment-guidelines

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • In resolve_external_urls, the local-development fallback hardcodes :3001, :4000/ui, and :8080; consider deriving these from the existing LITELLM_URL, LLAMA_SERVER_URL, and Langfuse configuration constants to avoid future port/config drift.
  • The default ROUTING_DOMAIN in resolve_external_urls and the PUBLIC_BASE_URL default in start-stack.sh both encode x570.vendeuvre.lan/vendeuvre.lan; consolidating these into a single shared configuration source would reduce the chance of subtle misalignment across components.
  • The new production deployment checklist in .agents/AGENTS.md hardcodes the boy user and specific host paths; consider parameterizing these (e.g., with placeholders or variables) so that the instructions are less tied to one machine/user and easier to adapt safely.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `resolve_external_urls`, the local-development fallback hardcodes `:3001`, `:4000/ui`, and `:8080`; consider deriving these from the existing `LITELLM_URL`, `LLAMA_SERVER_URL`, and Langfuse configuration constants to avoid future port/config drift.
- The default `ROUTING_DOMAIN` in `resolve_external_urls` and the `PUBLIC_BASE_URL` default in `start-stack.sh` both encode `x570.vendeuvre.lan`/`vendeuvre.lan`; consolidating these into a single shared configuration source would reduce the chance of subtle misalignment across components.
- The new production deployment checklist in `.agents/AGENTS.md` hardcodes the `boy` user and specific host paths; consider parameterizing these (e.g., with placeholders or variables) so that the instructions are less tied to one machine/user and easier to adapt safely.

## Individual Comments

### Comment 1
<location path="router/main.py" line_range="3093-3094" />
<code_context>
+    domain = os.getenv("ROUTING_DOMAIN") or "vendeuvre.lan"
+    if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host):
+        external_host = "localhost"
+    if not isinstance(external_netloc, str) or not re.match(r"^[a-zA-Z0-9.-]+(?::\d+)?$", external_netloc):
+        external_netloc = "localhost"
+
+    # Enforce strict domain validation to prevent loose substring match bypasses (e.g., attacker-vendeuvre.lan)
</code_context>
<issue_to_address>
**suggestion:** The netloc validation regex excludes valid forms like IPv6 literals and hostnames with non-standard characters, which may be too strict.

The `external_netloc` regex only accepts `[a-zA-Z0-9.-]` plus optional `:port`, which excludes valid cases like IPv6 literals (`[::1]:4000`), IDN/punycode hostnames, or reverse-proxy-modified netlocs. In those cases, silently forcing `localhost` can mask misconfigurations and change behavior unexpectedly. If the intent is mainly to filter clearly bad inputs, consider either relaxing validation (e.g., use `urlparse` and trust its hostname parsing) or explicitly logging/rejecting invalid values instead of defaulting to `localhost`.

Suggested implementation:

```python
import os
import re
import logging
from urllib.parse import urlparse

logger = logging.getLogger(__name__)

```

```python
        external_netloc = parsed.netloc or "localhost"
        external_scheme = parsed.scheme if parsed.scheme in ("http", "https") else "https"
    else:
        external_host = request.base_url.hostname or "localhost"
        external_netloc = request.base_url.netloc or "localhost"
        external_scheme = request.url.scheme if request.url.scheme in ("http", "https") else "https"

    domain = os.getenv("ROUTING_DOMAIN") or "vendeuvre.lan"

    # Basic sanity-check on external_host, but don't over-restrict valid hostnames;
    # fall back to the request base URL rather than silently forcing localhost.
    if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host):
        logger.warning(
            "Unexpected external_host %r, falling back to request.base_url.hostname (%r)",
            external_host,
            request.base_url.hostname,
        )
        external_host = request.base_url.hostname or "localhost"

    # Relax external_netloc validation: use urlparse so IPv6 literals, IDN/punycode,
    # and reverse-proxy-modified netlocs are supported. Log and fall back instead of
    # silently forcing localhost when invalid.
    if isinstance(external_netloc, str):
        parsed_netloc = urlparse(f"{external_scheme}://{external_netloc}")
        if not parsed_netloc.hostname:
            logger.warning(
                "Invalid external_netloc %r, falling back to request.base_url.netloc (%r)",
                external_netloc,
                request.base_url.netloc,
            )
            external_netloc = request.base_url.netloc or "localhost"
    else:
        logger.warning(
            "Non-string external_netloc %r, falling back to request.base_url.netloc (%r)",
            external_netloc,
            request.base_url.netloc,
        )
        external_netloc = request.base_url.netloc or "localhost"

```

- Ensure that `router/main.py` does not already define a `logger` with a different convention. If it does, reuse the existing logger instead of creating a new one.
- If your codebase uses a centralized logging configuration or helper (e.g., `from .logging import logger`), adapt the `logger` usage accordingly rather than importing `logging` and creating a new logger in this file.
- If the actual import section in `router/main.py` does not exactly match `import os` / `import re`, adjust the first SEARCH/REPLACE block to fit the existing imports while still adding `import logging` and `from urllib.parse import urlparse` plus the `logger = logging.getLogger(__name__)` line.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread router/main.py Outdated

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces dynamic URL resolution for the dashboard and external services (Langfuse, LiteLLM, Llama.cpp) by replacing hardcoded local paths with configurable placeholders and environment variables across pod.yaml, start-stack.sh, and README.md. It also re-enables the local Qwen model, updates the free models roster, and adds deployment documentation. Feedback is provided to ensure that external_host and external_netloc are kept in sync during validation to prevent mismatched URL generation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread router/main.py Outdated
Comment on lines +3091 to +3094
if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host):
external_host = "localhost"
if not isinstance(external_netloc, str) or not re.match(r"^[a-zA-Z0-9.-]+(?::\d+)?$", external_netloc):
external_netloc = "localhost"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If either external_host or external_netloc fails validation, they should both be reset to 'localhost' to keep them in sync. Otherwise, if external_netloc is reset to 'localhost' but external_host remains valid (e.g., matching the domain), is_valid_external will evaluate to True and construct a mismatched URL using 'localhost' as the netloc.

Suggested change
if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host):
external_host = "localhost"
if not isinstance(external_netloc, str) or not re.match(r"^[a-zA-Z0-9.-]+(?::\d+)?$", external_netloc):
external_netloc = "localhost"
if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host) or not isinstance(external_netloc, str) or not re.match(r"^[a-zA-Z0-9.-]+(?::\d+)?$", external_netloc):
external_host = "localhost"
external_netloc = "localhost"

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
litellm/config.yaml (1)

9-9: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Stale comment contradicts the re-enabled model.

Line 9 states local-qwen-3.6 (35B) disabled 2026-06-08 but the model is now re-enabled in every fallback chain (lines 41–63) and in the model_list (line 93). Update or remove this comment to avoid confusion.

📝 Proposed fix
   # -------------------------------------------------------------------------
-  # local-qwen-3.6 (35B) disabled 2026-06-08 — frees 23GB RAM/GTT.
+  # local-qwen-3.6 (35B) re-enabled 2026-07-08 — llama.cpp on port 8081.
   # -------------------------------------------------------------------------
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@litellm/config.yaml` at line 9, The comment for local-qwen-3.6 (35B) is stale
and contradicts the current configuration. Update or remove the note near the
top of config.yaml so it reflects that the model is re-enabled, and make sure
the wording matches the active fallback chains and model_list entries in the
same file.
🧹 Nitpick comments (2)
litellm/config.yaml (1)

93-99: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Hardcoded api_key in version-controlled config.

api_key: local-token is a plaintext credential in a tracked config file. If this is a dummy token for a local llama.cpp instance, consider documenting that explicitly or sourcing it from an environment variable to avoid setting a precedent for committing real secrets here.

🔐 Suggested approach
 - litellm_params:
     api_base: http://127.0.0.1:8081/v1
-    api_key: local-token
+    api_key: os.environ/LOCAL_QWEN_API_KEY
     model: openai/local-qwen-3.6
     request_timeout: 300
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@litellm/config.yaml` around lines 93 - 99, The config entry under the
local-qwen-3.6 model in litellm_params uses a hardcoded api_key, so update that
block to avoid committing a plaintext credential. Replace the literal token with
an environment-sourced value or a clearly documented placeholder for the local
llama.cpp setup, and keep the change localized to the existing local-qwen-3.6
configuration.
router/main.py (1)

21-29: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Duplicate urlparse import.

from urllib.parse import urlparse appears on both line 21 and line 25. The second import is redundant.

♻️ Remove the duplicate import
 from urllib.parse import urlparse
 from circuit_breaker import get_breaker
 from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator, RootModel
 from typing import Dict, Optional, Union
-from urllib.parse import urlparse
 
 LITELLM_URL = (os.getenv("LITELLM_ADMIN_URL") or "http://127.0.0.1:4000").rstrip("/")
 LLAMA_SERVER_URL = (os.getenv("LLAMA_SERVER_URL") or "http://127.0.0.1:8080").rstrip("/")
 LANGFUSE_HOST = (os.getenv("LANGFUSE_HOST") or "http://127.0.0.1:3001").rstrip("/")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@router/main.py` around lines 21 - 29, The module imports `urlparse` twice, so
remove the redundant `from urllib.parse import urlparse` and keep only one
import near the top of `router/main.py`. Make sure the remaining import is
sufficient for any `urlparse` usage elsewhere in the file and leave the other
imports untouched.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@router/main.py`:
- Around line 3141-3161: The local development fallback in the URL-building
logic is using hardcoded http:// URLs with external_host, which can produce
cleartext links to a non-local host when PUBLIC_BASE_URL is set without matching
ROUTING_DOMAIN. Update the fallback path in the URL construction branch to
either force localhost for the generated URLs or emit a warning when
external_host is not localhost, and keep the behavior consistent in the
parsed_lf, parsed_ll, and parsed_lm URL assembly.

---

Outside diff comments:
In `@litellm/config.yaml`:
- Line 9: The comment for local-qwen-3.6 (35B) is stale and contradicts the
current configuration. Update or remove the note near the top of config.yaml so
it reflects that the model is re-enabled, and make sure the wording matches the
active fallback chains and model_list entries in the same file.

---

Nitpick comments:
In `@litellm/config.yaml`:
- Around line 93-99: The config entry under the local-qwen-3.6 model in
litellm_params uses a hardcoded api_key, so update that block to avoid
committing a plaintext credential. Replace the literal token with an
environment-sourced value or a clearly documented placeholder for the local
llama.cpp setup, and keep the change localized to the existing local-qwen-3.6
configuration.

In `@router/main.py`:
- Around line 21-29: The module imports `urlparse` twice, so remove the
redundant `from urllib.parse import urlparse` and keep only one import near the
top of `router/main.py`. Make sure the remaining import is sufficient for any
`urlparse` usage elsewhere in the file and leave the other imports untouched.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5ea7333e-e10d-4d4a-8ab5-e882c7f0f71f

📥 Commits

Reviewing files that changed from the base of the PR and between 454593c and 8331d9b.

📒 Files selected for processing (8)
  • .agents/AGENTS.md
  • README.md
  • litellm/config.yaml
  • pod.yaml
  • router/agy_proxy.py
  • router/free_models_roster.json
  • router/main.py
  • start-stack.sh

Comment thread router/main.py
Comment on lines +3141 to +3161
else:
# Local development fallback: derive ports and paths dynamically from configuration constants
parsed_lf = urlparse(LANGFUSE_HOST)
parsed_ll = urlparse(LITELLM_URL)
parsed_lm = urlparse(LLAMA_SERVER_URL)

lf_port = f":{parsed_lf.port}" if parsed_lf.port else ""
ll_port = f":{parsed_ll.port}" if parsed_ll.port else ""
lm_port = f":{parsed_lm.port}" if parsed_lm.port else ""

lf_path = parsed_lf.path
ll_path = parsed_ll.path or "/ui"
if not ll_path.endswith("/ui") and not ll_path.endswith("/ui/"):
ll_path = ll_path.rstrip("/") + "/ui"
lm_path = parsed_lm.path

return (
f"http://{external_host}{lf_port}{lf_path}",
f"http://{external_host}{ll_port}{ll_path}",
f"http://{external_host}{lm_port}{lm_path}"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Local dev fallback uses http:// with potentially non-localhost external_host.

The else branch (local development fallback) hardcodes http:// and uses external_host, which may not be localhost when PUBLIC_BASE_URL is set but doesn't match ROUTING_DOMAIN. In that misconfiguration scenario, the function produces cleartext URLs to a remote host (e.g., http://some-host:3001) instead of either falling back to localhost or raising a clear error.

Consider either:

  1. Using "localhost" explicitly in the fallback branch instead of external_host, or
  2. Logging a warning when falling through to the local dev branch with a non-localhost external_host.
🔒 Suggested improvement: warn on non-localhost fallback
     else:
         # Local development fallback: derive ports and paths dynamically from configuration constants
+        if external_host not in ("localhost", "127.0.0.1"):
+            logger.warning(
+                "Domain validation failed for external_host %r (ROUTING_DOMAIN=%r); "
+                "falling back to local dev URLs with http://",
+                external_host, domain,
+            )
         parsed_lf = urlparse(LANGFUSE_HOST)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
else:
# Local development fallback: derive ports and paths dynamically from configuration constants
parsed_lf = urlparse(LANGFUSE_HOST)
parsed_ll = urlparse(LITELLM_URL)
parsed_lm = urlparse(LLAMA_SERVER_URL)
lf_port = f":{parsed_lf.port}" if parsed_lf.port else ""
ll_port = f":{parsed_ll.port}" if parsed_ll.port else ""
lm_port = f":{parsed_lm.port}" if parsed_lm.port else ""
lf_path = parsed_lf.path
ll_path = parsed_ll.path or "/ui"
if not ll_path.endswith("/ui") and not ll_path.endswith("/ui/"):
ll_path = ll_path.rstrip("/") + "/ui"
lm_path = parsed_lm.path
return (
f"http://{external_host}{lf_port}{lf_path}",
f"http://{external_host}{ll_port}{ll_path}",
f"http://{external_host}{lm_port}{lm_path}"
)
else:
# Local development fallback: derive ports and paths dynamically from configuration constants
if external_host not in ("localhost", "127.0.0.1"):
logger.warning(
"Domain validation failed for external_host %r (ROUTING_DOMAIN=%r); "
"falling back to local dev URLs with http://",
external_host, domain,
)
parsed_lf = urlparse(LANGFUSE_HOST)
parsed_ll = urlparse(LITELLM_URL)
parsed_lm = urlparse(LLAMA_SERVER_URL)
lf_port = f":{parsed_lf.port}" if parsed_lf.port else ""
ll_port = f":{parsed_ll.port}" if parsed_ll.port else ""
lm_port = f":{parsed_lm.port}" if parsed_lm.port else ""
lf_path = parsed_lf.path
ll_path = parsed_ll.path or "/ui"
if not ll_path.endswith("/ui") and not ll_path.endswith("/ui/"):
ll_path = ll_path.rstrip("/") + "/ui"
lm_path = parsed_lm.path
return (
f"http://{external_host}{lf_port}{lf_path}",
f"http://{external_host}{ll_port}{ll_path}",
f"http://{external_host}{lm_port}{lm_path}"
)
🧰 Tools
🪛 ast-grep (0.44.1)

[warning] 3157-3157: Do not make http calls without encryption
Context: f"http://{external_host}{lf_port}{lf_path}"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.

(requests-http)


[warning] 3158-3158: Do not make http calls without encryption
Context: f"http://{external_host}{ll_port}{ll_path}"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.

(requests-http)


[warning] 3159-3159: Do not make http calls without encryption
Context: f"http://{external_host}{lm_port}{lm_path}"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.

(requests-http)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@router/main.py` around lines 3141 - 3161, The local development fallback in
the URL-building logic is using hardcoded http:// URLs with external_host, which
can produce cleartext links to a non-local host when PUBLIC_BASE_URL is set
without matching ROUTING_DOMAIN. Update the fallback path in the URL
construction branch to either force localhost for the generated URLs or emit a
warning when external_host is not localhost, and keep the behavior consistent in
the parsed_lf, parsed_ll, and parsed_lm URL assembly.

Source: Linters/SAST tools

sheepdestroyer added a commit that referenced this pull request Jul 9, 2026
…idelines

chore: robust URL resolution, consolidated domain defaults, and generalized runbook (PR #244 fixes)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant